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ABSTRACT 


Maintenance  and  enhancement  of  application  software  consume  a major 
portion  of  the  total  life  cycle  cost  of  a system.  Estimates  of  the 
total  systems  and  programming  resources  consumed  range  as  high  as 
75-80%  in  each  category.  However,  the  area  has  been  given  little 
attention  in  the  literature.  To  analyze  the  problems  in  this  area 
a questionnaire  was  developed  and  pretested.  It  was  then  submitted 
to  120  organizations.  Respondents  totaled  69.  Responses  were  analyzed 
using  the  SPSS  statistical  package.  Hie  results  of  the  analysis  indicate 
that  1)  user  demands  for  enhancements  and  extensions  constitute  the  major 
perceived  problem  area;  2)  problems  of  a management  nature  are  viewed 
in  general  as  more  significant  than  technical  problems;  3)  the  use  of 
technical  productivity  aids  remains  limited;  however,  maintenance 
programmer  productivity  is  not  considered  by  management  to  be  a major 
problem . 
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1.  Introduction 

Hie  maintenance  and  enhancement  of  operational  application  software  systems 
is  frequently  viewed  as  a phase  of  lesser  importance  them  the  design  and 
development  phases  of  the  system  life  cycle.  Maintenance  and  enhancement 
are  generally  defined  as  activities  which  keep  systems  operational  and 
meet  user  needs  (see  for  example  Riggs  [16] ) . Types  of  maintenance  and 
enhancement  activities  have  been  developed  in  Swanson  [18] . This  work 
will  be  referred  to  later  in  more  detail. 


There  have  been  a number  of  estimates  of  the  amount  of  effort  that  goes 
into  maintenance  and  enhancement.  Riggs  [16]  cites  a range  of  40-60*  of 
total  systems  and  programming  resources.  Similar  figures  have  been  given 
in  [3],  [5],  [7],  [19].  An  estimate  as  high  as  75*  of  resources  has  been 
cited  in  [15].  A more  conservative  estimate  of  40*  has  been  given  in 
[8],  [9],  and  by  Boehm  [2].  Some  of  the  specific  problems  in  maintenance 
and  enhancement  have  been  the  effect  of  hardware  changes  (Boehm  [2])  and 


errors  introduced  with  modifications  (Rosy  [10] ) . 

Studies  involving  specific  software  systems  include  Stearns  [17]  and  the 
excellent  analysis  of  OS/MVT  by  Belady  and  Lehman  []].  Some 
interesting  ideas  on  maintenance  have  been  stated  by  Brooks  ([4]). 

Other  sources  which  take  a management  and  implementation  point  of 
view  include  [6],  [12],  and  [13]. 

The  purpose  of  this  paper  is  to  present  some  of  the  analysis  results  of 
a survey  of  organizations  involved  in  maintenance  and  enhancement. 
Section  2 presents  the  data  collection  process  employed  and  a profile 
of  respondents.  The  statistical  results  appear  in  section  3. 

Conclusions  are  given  in  section  4. 
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2.  Data  Collection 

This  section  summarizes  the  data  collection  process  as  well  as  the 
general  profile  of  respondents.  The  questionnaire  appears  in  [11]. 


The  process  of  data  collection  began  with  the  construction  of  an  initial 
questionnaire  and  a field  test  of  five  organizations . Refinements  were 
made  and  the  form  used  for  the  survey  finalized.  Some  120  organizations 
were  contacted  by  telephone  to  identify  the  proper  recipient  of  the  ques- 
tionnaire and  their  willingness  to  participate.  Questionnaires  were 
then  mailed  out  with  return  envelopes  and  postage  supplied.  Follow-up 
calls  were  made  if  no  response  was  received  within  two  weeks.  The  total 
number  of  responses  was  69.  This  is  a substantial  percentage  considering 
the  length  and  the  depth  of  the  questionnaire. 

The  questionnaire  is  composed  of  two  parts.  Part  I deals  with  the 
systems  and  programming  department  and  contains  12  questions  in  the 
following  areas : 

- industry  category 

- annual  budget  for  software  and  hardware 

- number  of  personnel  in  department  (systems  analysts  and 

programmers  as  well  as  aggregate) 

- division  of  tasks  among  staff  in  maintenance  and  new 
application  work,  and  in  analysis  and  programming 

- management  structure 

- current  precentage  of  effort  in  maintenance 


- relative  importance  of  maintenance  compared  to  development 

- reallocation  of  effort  between  maintenance  and  development, 
given  hypothetical  budget  increases  and  decreases 

- evaluation  of  adequacy  of  current  levels  of  staffing 

The  second  part  of  the  questionnaire  deals  with  the  application  software 
undergoing  maintenance  and  enhancement.  Respondents  were  asked  to  select 
a system  which  has  been  operational  for  at  least  one  year,  represents 
a significant  investment  of  time  and  effort,  and  is  of  fundamental 
importance  to  the  organization.  For  this  system  they  answered  38 
questions  on  the  following  topics: 

- name  of  system,  function,  and  end  users 

- number  of  personnel  in  user  groups* 

- number  of  personnel  in  user  groups  actively  involved  in  the 
system  processing  cycle* 

- date  system  became  operational 

- number  of  programs  maintained  and  number  of  source  language 
statements  broken  down  by  language* 

- distribution  of  source  statements  according  to  origination 
year* 

- percentage  of  system  dealing  with  on-line  processing* 

- total  number  of  machine  language  statements* 

- harchrare/software  environment  of  system 

- use  of  distributed  processing  and/or  data  base  managements 
systems 

- number  of  files,  average  size  of  data  base*,  percentage  of 
data  base  updated  by  time  period* 


- number  and  form  of  predefined  user  reports* 

- productivity  tools  used  in  development 

- time  spent  on  maintenance* 

- division  of  effort  among  types  of  maintenance  activities* 

- percentage  of  maintenance  effort  on  on-line  programs  and  in 
communication  with  user* 

- number  of  people  involved  in  maintenance  of  the  system,  the 
levels  of  their  programming  experience,  when  they  began  to  work 
on  the  system,  and  task  allocation  in  terms  of  analysis 

and  programming 

- formal  procedures  for  maintenance  request  handling,  number 
of  requests  received 

- formal  procedures  for  making  changes  to  programs,  and  number 
of  changes  made 

- formal  procedures  for  trouble  reporting 

- existence  of  auditing,  documentation,  cost  accounting 
procedures  and  chargeback  methods 

- problem  areas  in  maintenance  of  the  system 

In  the  above  list,  for  the  items  marked  with  an  asterisk  (*)  the 
respondents  also  answered  the  question:  "Check  the  applicable 

statement:  the  above  answer  is:  reasonably  accurate,  based  on 

good  data;  a rough  estimate,  based  on  minimal  data,  or  an 

estimate,  not  based  on  any  data." 


5 


r r 

|- 

3.  Analysis  Results 

. 
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This  section  is  organized  into  the  following  categories:  profile 

of  respondents,  tools  and  techniques  employed,  evaluation  of  maintenance, 

e 

and  interelationship  of  variable. 


Profile  of  Respondents 


Bach  respondent  was  asked  to  indicate  the  industry  segment  of  their 
organization.  A classification  of  the  responded  indicated 

manufacturing  - 27  (39.1%)  and  nonmanufacturing  - 42  (59.4%). 


Several  questions  were  asked  of  their  data  processing  equipment  and 
annual  cost.  The  response  on  equipment  was  similar  to  the  division 
of  the  market  and  was  IBM  (73.9%),  Burroughs  (8.7%),  Honeywell  (5.8%), 
NCR  (4.3%),  Univac  (4.3%),  and  others  (2.8%). 


The  distribution  of  annual  budget  for  hardware  is  given 
in  Table  I. 


TABLE  I:  BUDGET  DISTRIBUTION  FOR  EQUIPMENT 

Budget  ($1,000* s)  Percentage 


Under  250 
250  - 500 
500  - 1000 
1000  - 2000 


14.5% 

15.9 

14.5 

11.5 


J 


Over  2000 


40.6 
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Several  questions  were  asked  on  how  development  and  maintenance  effort 
would  be  redistributed  if  the  systems  and  programming  staff  were 
increased  or  reduced  by  certain  percentages.  The  results  are 
summarized  below  (Table  II)  and  indicate  that  most  additional 
resources  would  go  to  new  development.  Also  as  expected,  most 
budget  reductions  would  occur  in  new  development. 


TABLE  II.  EFFECT  OF  BUDGET  DIFFERENCE 
Budget  Change  Distribution  of  Change 

Maintenance  and 

New  Development  Enhancement  Other 


Mean 

Std.  Dev. 

Mean 

Std.  Dev. 

10% 

Increase 

6.6% 

3.4% 

3.1% 

3.3% 

.3% 

25% 

Increase 

15.6 

6.7 

8.3 

6.2 

1.1 

10% 

Decrease 

7.3 

3.3 

2.4 

3.3 

.3 

25% 

Decrease 

17.1 

6.6 

6.8 

6.5 

1.1 

In  this  table  the  size  of  the  standard  deviation  is  somewhat  unexpected. 
From  the  questionnaire  data  it  appears  that  with  a 10%  increase 
there  are  some  organizations  which  would  budget  the  entire  increase 
for  maintenance  and  enhancement.  At  the  25%  level  this  group  would 
allocate  more  to  new  development. 

One  major  management  issue  involving  maintenance  centers  on  methods 
for  charging  back  costs  to  the  user.  Of  the  sample  almost  60% 

(59.4%)  do  not  charge  back  the  use  for  operations  or  for  main- 
tenance and  enhancement  work.  Of  the  respondents  using  a charge- 
back  method  90%  chargeback  both  computer  and  personnel  expenses. 


Questions  of  budgeting  levels  lead  to  the  issue  of  adequacy  of 
staffing  levels.  Most  felt  that  they  were  somewhat  understaffed. 
The  reponses  were: 


Substantially  understaffed  8.7% 

Somewhat  understaffed  60.9 

Properly  staffed  26.1 

Somewhat  overstaffed  4.3 


Several  observations  can  be  made  on  the  organization  of  the  respondents. 
When  asked  for  a breakdown  between  analysts  and  programmers,  most 
respondents  have  staff  members  assigned  to  both  maintenance  and  enhance- 
ment as  well  as  new  development  work.  Programming  is  treated  as  a 
separate  activity  by  only  40.6%  of  respondents. 

Of  the  respondents  68.1%  treated  maintenance  and  enhancement  as  a separate 
activity  from  development.  In  terms  of  annual  personnel  hours  allocated 
to  maintenance  and  enhancement,  and  new  development,  the  results  were: 
Maintenance  and  enhancement  48.0% 

New  development  46.1 

Other  activities  5.9 

This  is  among  the  lower  estimates  for  maintenance  effort  cited  in  the 
literature.  However,  there  were  a significant  number  of  cases  (over  20%) 
tht  allocated  85%  of  their  effort  to  maintenance. 
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Within  the  maintenance  effort  the  breakdown  given  was: 


Category 


Activities 


Relative  Frequency 


Corrective  Emergency  fixes,  routine  debugging 


Adaptive  Accommodation  of  changes  to  data  inputs 

and  files,  and  to  hardware  and  system 
software 


17.4% 


18.2% 


Perfective  User  enhancement,  improved  documentation,  60.3% 
recording  for  computational  efficiency 

Other  4.1% 

In  this  setting  perfective  maintenance  is  by  far  the  biggest  area  of  effort. 

This  will  be  further  supported  later  in  Table  V which  indicates  that  user 

demands  for  enhancements  and  extensions  are  perceived  by  management  to 

be  the  biggest  problem. 


Several  questions  were  asked  on  accounting  for  user  requests  and  system 
problems  as  well  as  auditing.  It  was  found  that  68.1%  logged  and  docu- 
mented maintenance  and  enhancement  requests.  A lower  percentage  (55.1%) 
logged  and  documented  operational  problems  with  the  application  system. 
When  asked  whether  a formal  audit  of  the  application  system  is  made 
periodically,  only  37.8%  responded  yes. 


Tools  and  Techniques  Employed 

Respondents  were  asked  to  distribute  the  percentages  of  source  code  lines 
by  language.  As  expected,  the  preponderance  was  in  COBOL  and  Assembler. 
The  distribution  was: 


COBOL 


58.1% 


Assembler 


18.5% 


FORTRAN 


ALGOL 


10.2% 


Other 


6% 
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K somewhat  frequently  made  assertion  in  the  literature  is  that  pro- 
ductivity tools  in  design  and  programming  are  not  yet  Ndidely  employed 
in  practice.  This  is  substantially  borne  out  in  the  percentages 
given  in  Table  III.  In  Table  III  the  most  frequently  used  tool  is 
decision  tables  (46.4%).  Other  tools  in  use  by  at  least  30%  of  respon- 
dents included  test  data  generators,  on-line  programming  and  chief 
programmer  teams.  It  is  interesting  to  note  that  approximately  one 
quarter  of  the  sample  indicated  that  they  use  structured  programming. 
Responses  other  than  those  in  Table  III  include  modular  programming, 
top-down  testing,  on-line  simulator,  copy  library,  and  technical  design 
review.  It  should  be  noted  that  the  percentages  from  Table  III  reflect 
operational  application  systems;  for  systems  currently  being  developed 
the  figure  might  be  somewhat  higher. 


In  several  related  questions  few  of  the  respondents  indicated  use  either 
date  base  management  systems  (21.7%)  or  distributed  processing  (4.3%). 

Interestingly,  maintenance  programmer  productivity  is  not  regarded  by 
management  as  a significant  problem,  as  will  be  shown  later.  This  would 
seem  to  indicate  that  movement  toward  a widespread  use  of  productivity 
aids  will  continue  to  be  slow. 

Evaluation  of  Maintenance 

The  respondents  were  asked  to  contrast  the  relative  importance  of  main- 
tenance with  new  system  development  within  their  organizations.  The 
relative  frequency  appears  in  Table  IV.  It  indicates  most  view  main- 
tenance as  more  important  than  new  development.  More  strikingly,  few 
view  new  system  development  as  more  important. 


-rawT.*-  hi.  USE  OF  DESIGN  AND  PROGRAMMING  AIDS 


Tool 

Relative 

Yes 

Frequen 

N 

Decision  Tables 

46.4 

5 

Test  data  generators 

36.2 

6 

Chief  programmer  team 

30.4 

6 

On-line  programming 

30.4 

6 

Data  base  dictionary 

26.1 

7 

Structured  programming 

24.6 

7 

Structured  walk- thru 

17.4 

8 

Automatic  flowcharting 

10.1 

8 

HIPO 

7.2 

9 

ISDOS  (Automated  design  aid) 


4.3 
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SABLE  IV.  IMPORTANCE  OF  MAINTENANCE  COMPARED 
TO  NEW  SYSTEM  DEVELOPMENT 


Maintenance  and  Enhancement 


Percentage 


Ey  far  more  important 
Somewhat  more  important 
Equal  importance 
Somewhat  less  important 
By  far  less  important 


33.3% 


Rlpondents  were  further  asked  to  rank  possible  problem  areas  in  maintenance, 
ttiis  is  summarized  in  Table  V.  The  table  columns  are  arranged  by  problem  area* 
statistics,  and  relative  frequency.  The  statistics  are  based  on  the  coding: 
1-not  a problem,  2-somewhat  minor  problem,  3-minor  problem,  4-somewhat  major 
problem,  5-major  problem.  Items  narked  with  an  asterisk  indicate  technical 
problem  areas . 


Problem  Area 


tank 


Statistics 


std. 

Mean  Median  Dev. 


Relative  Frequen 


User  demands  for  en- 
. han cements , extens. 

3.42 

Quality  of  syst.. 
docum. * 

2.99 

Competing  demands  on 
aaint.  personnel 
time 

2.95 

Quality  of  original 
programs* 

2.94 

Meeting  scheduled 
commitments 

2.79 

Lack  of  user  under- 
stand. of  syst. 

2.66 

Availability  of  main, 
program,  personnel 

2.66 

Adequacy  of  syst. 
design  spec.* 

2.52 

Turnover  of  mainten. 
personnel 

2.46 

Unrealistic  user 
expectations 

2.45 

Processing  time  of 
system* 

2.31 

Forecast,  personnel 
requirements 

2.30 

Skills  of  maint. 
personnel* 

2.20 

Changes  to  hardware 
and  software* 

2.14 

Budgetary  pressures 

2.09 

Adherence  to  program, 
stds.  in  maint.* 

2.08 

Data  integrity* 

2.06 

Motivation  of  maint. 
personnel 

2.03 

Applic.  run  fail- 
ures* 

2.00 

Maint.  programming 
productivity 

2.00 

Hardware  and  soft- 
ware reliability* 

1.91 

Storage  requiremts.* 

1.88 

Mgmt.  Support  of 
system 

1.87 

Lack  of  user  interest 
in  system 

1.86 

Some- 

Some- 

what 

what 

Mo 

Minor 

Minor  Major 

Major  Res 

¥ 


issues  (quality  of  original  system  and  its  documentation)  and  one 


Management  issue  (competing  demands  for  personnel  time) . Frequently 


mentioned  included  quality  of  operations  personnel,  turnover  in  user 


organization , high  learning  curve  due  to  large  system,  and  retaining 


than  technical  issues.  This  would  serve  as  a guide  in  efforts  to 


for  each  respondent,  the  average  rating  was  computed  for  technical  and 
management  areas  for  each  respondent.  The  Mann  Whitney-Wilcoxan  and 
sign  tests  were  selected  to  test  the  hypothesis  that  the  distribution  of 
the  average  response  of  each  category  was  the  same.  These  tests  do  not 
depend  on  actual  scores  but  relative  ratings.  For  the  Mann  Whitney- 
Wilcoxan  test  the  hypothesis  was  rejected  at  the  a * .10  level.  For 
the  sign  test  it  was  rejected  at  the  a - .01  level.  Both  results  indicated 


higher  values  for  the  management  areas 
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A -second  hypothesis  is  that  the  r^xsnse  to  the  problem  of  user  demands 
fax  enhancement  and  extension  is  significantly  larger  than  average 
■fax  all  problem  areas.  The  same  nonpar aae trie  tests  were  applied  and 
the  hypothesis  of  the  same  distribution  was  rejected  at  the  a - . 10  level. 
'Shis  Indicates  user  demands  are  more  of  a problem  than  other  areas. 


As  was  mentioned  in  Section  2,  seme  of  the  questions  were  followed  by 
questions  on  the  quality  of  data  on  which  the  answer  was  based.  The 
results  are  summarized  by  average  and  relative  frequency  in  Table  VI. 

An  asterisk  indicates  technical  subjects.  A question  here  is  whether 
there  is  less  data  available  for  management  type  questions  than  for 
technical  type  questions.  The  results  indicates  respondents  had  firmer 
data  for  technical  management  types  of  questions.  The  statistical  test 
was  to  test  that  the  average  responses  to  the  management  question  are 
based  on  data  of  a quality  average  equal  to  that  of  responses  for  tech- 
nical questions.  The  nonpar ame trie  tests  applied  were  the  sign  test 
and  the  Mann-Whitney  Wilcoxan  test.  Both  tests  rejected  the  hypothesis 
at  the  a = .10  level. 

Similar  tests  (at  a = .10  level)  indicated  that  respondents  knew  more 
about  effort  in  maintenance  and  enhancement  in  general  than  specific 
tasks  within  maintenance  and  enhancement. 


I 

I 


A 
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Answer  Based  on 


Seasonably 

Accurate  Minimal  No 

Data  Data  Data 


-4total  mach.  lang.  statements* 

X&strib.  of  source  code  overtime* 

Mo.  of  source  lang.  statements* 

% personnel  in  input/output 

% time  period  update  data  base* 

Sice  of  data  base* 

No.  personnel  in  user  organ.* 

% hrs.  by  activity  in  maintenance 

% hrs.  for  user  commun.  in  mainten. 

Hrs.  spent  on  maintenance 

Form,  freq.  of  user  reports* 

Number  of  user  reports* 

% of  statements  used  in  on-line 
process. 

Lang,  used* 

No.  programs* 

% hrs.  for  maintenance  of 
on-line  programs 


*Indicates  question  of  a technical  nature 


Interrelation  of  variables 


The  previous  subsections  of  this  section  were  concerned  with  responses  to 
individual  questions.  This  subsection  exanines  the  responses  for  inter- 
relationships between  response  items. 

The  analysis  indicated  that  system  characteristics,  unit  maintenance 
time  and  other  factors  are  not  highly  correlated.  Weighted  maintenance 
time  is  measured  as  personnel  time  in  maintenance  and  enhancement  divided 
by  the  total  number  of  source  statements  maintained.  The  highest  cor- 
relations obtained  were  between  the  number  of  programs  in  the  system 
and  the  number  of  predefined  user  reports  on  a daily  basis  (correlation 
coefficient  of  .69)  and  between  the  number  of  predefined  user  reports 
and  unit  maintenance  time  (correlation  coefficient  of  . 58) . The  factors 
included  in  the  correlation  included  unit  maintenance  time,  number  of 
personnel  in  user  units,  percent  of  primary  users  engaged  in  input/output, 
size  of  data  base,  number  of  files,  number  of  programs,  nuaber  of  predefined 
user  reports  (total  and  daily),  date  system  became  operational,  and  per- 
centage of  time  spent  in  communication  with  user.  Multiple  regression 
and  principal  Component  analysis  were  attempted  with  the  dependent  unit 
maintenance  time.  The  variables  were  added  stagewise  in  the  following 
order:  number  of  predefined  user  reports,  number  of  personnel  in  primary 

user  units,  nuaber  of  programs,  and  date  the  system  became  operational. 

It  has  been  suggested  that  after  a system  becomes  operational,  the  percent 
of  effort  in  emergency  fixes  and  routine  debugging  declines  with  time.  It 
then  increases  as  enhancement  work  changes  the  system  and  declines.  Some 
support  for  this  was  found  in  the  regression  analysis. 
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Conclusion* 

From  the  analysis  of  the  survey  data  several  tentative  conclusions 
are  suggested.  It  should  be  emphasized  that  these  are  based  on  the 
liadted  sample.  The  conclusions  are: 

o Maintenance  and  enhancement  consume  much  of  the  total  resources 
of  systems  and  programming  groups,, 
o Maintenance  and  enhancement  tend  to  be  viewed  by  management  as 
at  least  somewhat  more  important  than  new  application  software 
development, 

o In  maintenance  and  enhancement,  problems  of  a management 
orientation  tend  to  be  more  significant  than  those  of  a 
technical  orientation, 

o User  demands  for  enhancements  and  extensions  constitute  the 
most  important  management  problem  area, 
o The  use  of  productivity  aids  in  application  software  development 
remains  limited.  However,  maintenance  programmer  productivity 
is  not  considered  by  management  to  be  a major  problem. 

Overall  more  attention  should  be  given  to  management  problems  associated 
with  maintenance.  In  practice,  maintenance  work  should  be  categorized 
to  permit  the  gathering  of  more  detailed  management  information.  Project 
reporting  systems  should  be  detailed  with  respect  to  the  type  and  tasks 
of  maintenance  and  enhancement. 

The  handling  of  user  request  for  enhancements  should  be  examined  to 
determine  means  of  better  evaluating  and  satisfying  requests. 
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